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ABSTRACT 


This paper provides an example of a global data sharing project with an educational transformation 
agenda. This agenda shaped both the design of the shared dataset and the experience of sharing 
the common dataset to support multiple perspective inquiry and enable integrative and critically 
reflexive research-to-practice dialogue. The shared dataset contains multiple disciplinary instances 
of design review conversations - digital videos of conversations between those who give and those 
who receive feedback, guidance, critique, or mentoring during a design review event. International 
design researchers were invited to apply their expertise to the dataset and participate face-to-face 
in a multi-day symposium for catalyzing conversations connecting design thinking, teaching, and 
learning. In this paper we describe this data sharing project, highlighting (1) the principles that guided 
the design of the shared dataset, (2) challenges encountered with implementing the project, and 
(3) the approach for broadening the dataset experience from sharing to enabling learning partner¬ 
ships. We conclude with insights from this experience for improving methodologies for global data 
sharing and designing data sharing for educational transformation. 

Key words: data sharing, multiple perspective inquiry, learning partnerships, educational transfor¬ 
mation, design thinking 


INTRODUCTION 


Integrating knowledge about how people learn with the practice of teaching and the design of learn¬ 
ing environments is a well-known problem that has evaded the best of intentions (Borrego, Froyd and 
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Hall, 2010; Henderson, Finkelstein and Beach, 2010; Kezar and Eckel, 2000). Literature on approaches 
to transforming engineering education focus on the uptake of research, building a foundation for 
evidence-based practice, developing reflective teachers and learning communities, enacting policy, 
developing a shared vision, taking a complex social systems perspective, and enabling a community 
of practitioners and researchers collaborating in support of a continuous cycle of educational practice 
and research (ASEE, 2012; Henderson, Finkelstein and Beach, 2010; Lattuca and Stark, 2009; NAE, 2012; 
Siddiqui and Adams, 2013; Siddiqui, 2014). Many of these emphasize the importance of dialogue, criti¬ 
cal reflection, and discourse, which are the essence of transformative learning approaches to change 
(Baxter Magolda and King, 2004; Kegan and Lahey, 2009; Lysaker and Furuness, 2011). 

In the engineering education community, many point to the challenges of filling in or bridging 
gaps in the current knowledge base such as an emphasis on thinking in terms of local level change 
as compared to system level change (McKenna, Froyd and Litzinger, 2014), what engineering should 
look like as compared to guidance on how to achieve that vision (Besterfield-Sacre, Cox, Borrego, 
Beddoes and Zhu, 2014), and a broaden adoption perspective that motivates a drive for generaliz- 
able research as compared to a social reconstruction or transformative learning perspective that 
motivates a drive for contextualized research (Siddiqui and Adams, 2013). The community requires 
a stronger empirical foundation on how people learn engineering or become engineers that can 
provide discipline-based evidence for guiding engineering education practice (NAE, 2012). 

Other research illustrates challenges associated with understanding faculty motivations and 
beliefs and how these shape approaches to accessing, sharing, and adopting teaching practices. 
Research on STEM education transformation indicates that faculty change their practice through 
naturalistic “pull” dissemination strategies - adapting practices directly from other practitioners via 
discussion and observation and serendipitous insights (Fincher, Finlay, Sharp, Falconer and Richards, 
2012; Fincher and Tenenberg, 2007), learning communities (Cox, 2004) and learning partnerships 
(Wildman, 2014), social neighborhoods (Siddiqui, 2014; Spalter-Roth, Mayorova and Shin, 2011), and 
reflective practice (Henderson, Beach and Finkelstein, 2011; Turns, Adams, Martin, Linse and Atman, 
2002). There is a growing body of research that illustrates the complex relationship between faculty 
beliefs about teaching and learning and successful adoption of teaching practices (Henderson et al, 
2011) and that robust evidence may be necessary but insufficient for driving change (Matusovich, 
Paretti, McNair and Hixson, 2014). In two different studies, Matusovich et al (2014) and Siddiqui 
(2014) found that educators and administrators hold personal perspectives about improving engi¬ 
neering education that may conflict or complement collective values or other people’s beliefs. Also, 
researchers may not clearly articulate the practical uses for their work (Turns, Paine, Sattler and 
Munoz, 2012), in effect placing research outside conversations about practical impact. While many 
note the importance of providing faculty with easily accessible and useful resources for implementing 
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teaching innovations (Lattuca and Stark; 2009; Seymour, DeWelde and Fry, 2011), research sug¬ 
gests that teachers struggle with reconstructing generalized materials (even when well-crafted) to 
fit their own teaching needs (Hutchinson and Huberman, 1994), and recognizing solutions to their 
own educational problems when searching through publicly available resources (Fincher et al, 2012). 

These challenges and insights have similar manifestations in design education, which spans many 
domains of practice from engineering, through architecture and industrial design, communications 
and into more artist expressions of design. Recently, the adoption of “design thinking” has afforded 
the opportunity to have a common language for talking about design research and design teaching. 
Although decades of design research have produced significant findings, there are still enormous 
gaps in our understanding of the particular knowledge designers (in any discipline) possess, what 
happens in the designer’s mind, and how designers develop ambidextrous mindsets for innovation 
(e.g., Cross, 2010; Goldschmidt and Badke-Schaub, 2009; Lande and Leifer, 2010). As Cross (2010) 
notes, the community lacks a working synthesis of design thinking, which can create a significant 
barrier for connecting research and practice. Other barriers include the challenges associated 
with applying research on design thinking to improve design teaching and student learning. For 
example, design teachers may be more likely to turn to colleagues than access existing research- 
based practices (Pembridge and Paretti, 2010) or struggle with navigating and translating scholarly 
publications on design research (Turns et al, 2002). In addition, translating between research and 
practice is often done in isolation, rather than researchers and practitioners working together to 
generate use-inspired connections. 

In this paper we describe a global data sharing project framed, designed, and implemented through 
an educational transformation lens. The 10th Design Thinking Research Symposium (DTRS 10) was a 
data sharing project designed to address key challenges associated with transformation in the broad 
domain of design education (Adams, 2016a). A central component of DTRS 10 was sharing a common 
data set of design review conversations - digital video recordings of diverse conversations between 
those who give and those who receive feedback, guidance, critique, or mentoring during a design 
review event. Design researchers from around the world were invited to apply their expertise to the 
dataset and share their insights face-to-face through four environments: symposium presentations 
and related discussions, artistic inquiry and improvisation-based reflective practices, a Research-to- 
Practice workshop for linking research presentations to practical educational actions, and cyber-enabled 
networking to support conversations before and after the event (Adams, 2016a). As such, one goal of 
DTRS 10 was to overcome a fragmented knowledge base that impedes the creation of evidence-based 
teaching resources by offering a shared data set experience explicitly designed to enable integrative 
syntheses inclusive of multiple paradigms and critical variations in teaching approaches. A second 
goal was to utilize learning partnership (Baxter Magolda and King, 2004) and collaborative inquiry 
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(Kasl and Yorks, 2002) principles to catalyze continual research-to-practice dialogue (ASEE, 2012), 
critical reflection and discourse among DTRS 10 participants. Targeting a design thinking phenom¬ 
enon - design review conversations - that has relevance for researchers, teachers, and practitioners 
offered additional motivations for catalyzing conversations connecting features of design thinking, 
design learners, and design learning environments. Within the learning partnership framework, a third 
goal was to empower DTRS 10 participants with bringing together their own identities as researchers, 
teachers, and practitioners. In other words, to help participants articulate implications of their research 
for practice (Turns, Paine, Sattler and Munoz, 2012) and apply research into their own teaching practice. 
As such, this project offers a complementary, but different, approach to educational transformation. 

This DTRS 10 data sharing experience is situated in a larger history of data sharing projects. In 
particular, the Design Thinking Research Symposium (DTRS) series includes three other shared 
dataset projects, each with a common goal of advancing an understanding of the particular knowl¬ 
edge designers’ possess and how they acquire this knowledge (Adams, 2016a; Adams, Cardella and 
Purzer, 2016; Cross, Christiaans and Dorst, 1996; Dorst, 1995; McDonnell and Lloyd, 2009a; 2009b; 
Rodgers, 2012; 2013). While DTRS 10 continues this tradition, the scale of the shared dataset is un¬ 
precedented, moving from 4 video recordings across two contexts and two time periods to 86 video 
recordings across 6 contexts with most covering multiple points over a single design timeline. The 
design of the symposium experience was broadened to include activities to support dialogue and 
critical reflection, and catalyze connections between research findings and teaching approaches. 
Other data sharing projects have had similar goals and breadth of participation (Adams, Fincher et al, 
2007; Fincher and Tenenberg, 2006; Petre, van der Hoek and Baker, 2010), but at a lesser scale. 

The following sections describe our experiences with designing and implementing DTRS 10 with 
a specific focus on the principles underlying our approach to data sharing, and lessons learned from 
problems encountered. We focus on three aspects of DTRS 10: (1) the intentional design for data 
sharing, (2) implementing data sharing, and (3) moving from sharing to learning partnerships. We 
conclude with implications for large scale and global data sharing, in particular data sharing projects 
that adopt an educational transformation agenda. Additional details about DTRS 10 are available in 
Analyzing Design Review Conversations (Adams & Siddiqui, 2016). 


INTENTIONAL DESIGN FOR SHARING 


A unique aspect of the DTRS 10 data sharing project is that the dataset was designed for shar¬ 
ing, it is not a data reuse or data platform project. In other words, decisions about what the dataset 
would contain and how it would be shared and used occurred together, often iteratively. 
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Design for connection, emergence, and critical inquiry 

The shared dataset contains multi disciplinary instances of design review conversations : digital 
videos of conversations between those who give and those who receive feedback, guidance, cri¬ 
tique, or mentoring during a design review event. Across disciplines, design reviews are a prevalent 
and fundamental pedagogy for helping student designers develop and demonstrate design think¬ 
ing expertise (Dym et al, 2005; Huet, Culley, McMahon and Fortin, 2007; Goldschmidt, 2002). They 
make the design thinking of design coaches and students visible. During a design review, coaches 
notice problematic and promising aspects of a designer’s work - drawing on repertoires from similar 
situations to anticipate and provide advice on problems students may encounter, connect student 
work to historical and cultural precedent, point out features of a design that could be wrong or im¬ 
proved, praise design work pointing to particular insights and choices, and push students to justify 
their ideas about "good design” and reflect on their thought processes (Adams, 2016). In this way, 
design students are supported in making sense of design review experiences in ways that allow 
them to construct their own design thinking repertoire and evolving design identity. Design reviews 
also illustrate the mechanism of reflective practice (Schon, 1993) and formative feedback, which is 
critical in helping students succeed particularly for complex problem-solving activities (Zessoules 
and Gardner, 1991). 

The shared dataset contains 86 digital video recordings (ranging from 5 to 75 minutes) with time- 
stamped transcripts and 126 related work products (e.g., interim and final presentations, reports, 
prototypes, course syllabi, and project design briefs) (Adams, 2016a). As illustrated in Figure 1, the 
dataset was designed to provide multiple entry points to enable new lines of inquiry while evok¬ 
ing connection, disorientation, and transformation (Mezirow, 2009). These multiple entry points 
include variations in: review structures (individual/group, informal/formal, juried), interaction mo¬ 
dalities (text, speech, objects, gestures), design phases (longitudinal data from first to final review), 
disciplinary lenses and interdisciplinary settings (choreography, entrepreneurial design, industrial 
design, mechanical engineering, service learning design), and design coaches (instructors, experts, 
stakeholders, and peers). 

Three conceptual frameworks guided the design of the dataset - scholarship of integration 
(Boyer, 1990), multiple perspective methodology (Mitroff and Linstone, 1993), and complexity 
theory (Davis & Sumara, 2006). With reference to a scholarship of integration (Boyer, 1990), the 
choice of student design reviews was an intentional decision to focus a research lens on a phe¬ 
nomenon that sits in a middle space between research and practice. For design teachers, design 
reviews represent a pedagogy that is familiar to what they do and what they experience with their 
students; for design researchers, design reviews are an underdeveloped area of research that is 
theoretically and methodologically rich (Adams, 2016a). Meeting the goals of a scholarship of 
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Figure 1. The DTRS 10 shared data set (from Adams, 2016b; Adams, Cardella, and Purzer, 
2016). 


integration means creating a dataset that would support bringing disparate research into a trans- 
disciplinary perspective of larger intellectual patterns (Adams et al, 2011; Solis, Strong, Adams, 
Turns & Crismond, 2016). 

We used multiple perspective methodology to design a dataset that could support unbounded 
systems thinking (Adams, 2016a). Design settings and data collection techniques (visual, textual, 
artifactual) were selected to support an interparadigmatic mix of thought (Linstone et al., 1981) (e.g., 
rational, interpretivist, emancipatory) across disciplines (choreography, entrepreneurial design, in¬ 
dustrial design, mechanical engineering, and service learning design) and design thinking paradigms 
(aesthetic, functional, technical, entrepreneurial, and human-centered). Here, an interparadigmatic 
mix means that the data set embodies diverse paradigms, or worldviews, about design thinking. As 
such, the data set is designed to offer DTRS 10 participants with rich opportunities to investigate 
diverse (and potentially conflicting) design thinking paradigms, even though the data set is limited 
to a single US context. From a learning partnership standpoint, a benefit of an interparadigmatic 
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mix is that it supports inclusiveness and productive conflict that could bring assumptions to the 
surface and set in play a process for integrative and transformative thinking of symposium partici¬ 
pants (Mitroff and Linstone, 1993; Mezirow, 2000). 

We used complexity theory to design a dataset that could enable conditions of emergence and 
attend to critical variations of design review situations (Adams, 2016a), in particular the complemen¬ 
tary tensions of randomness and coherence as well as diversity and redundancy (Davis and Sumatra, 
2006). To maximize the richness of the dataset, every effort was made to collect data from the same 
students or teams over time and over multiple design phases (e.g., problem formulation, concept 
review, stakeholder review, final review). Research on critical variations of design review situations 
that affect the practice of critiquing informed the strategic selection of design review situations that 
include variations in terms of setting, types of coaches, and modalities (speech, written comment, 
drawing, gestures, and artifacts) (Oh, Ishizaki, Gross and Do, 2012). 

Interparadigmatic mix among DTRS 10 participants 

Multiple perspective methodology also informed the process for engaging a broad interparadig¬ 
matic mix among DTRS 10 participants. Invitations to DTRS 10 were distributed by (1) using social 
networks of the organizing committee to ensure broad disciplinary representation, (2) providing 
permission to the first wave of invitations to forward the invitation to others, (3) using active internet 
lists in the design thinking research community (e.g., the Design Research Society website), and 
(4) distributing invitations to past DTRS participants. Over 40 research teams from 13 countries 
(Australia, Canada, China, Denmark, England, Germany, Holland, Israel, Japan, Portugal, Spain, South 
Africa, Turkey, and the United States) submitted formal commitments to participate and completed 
Data Use Agreements. To illustrate diversity of perspectives, proposals identified over 115 design 
thinking frameworks and over 20 different methodological frameworks (e.g., behavioral economics, 
phenomenology, discourse analysis, naturalistic decision making, activity theory, variation theory, 
critical feminist theory, networked design practice, interaction analysis, data mining, etc.). Twenty- 
eight papers were selected, involving 83 individuals. Participants represented disciplines of art 
(choreography, technology-based art, and graphical design), architecture, business, cognitive and 
learning sciences, communication, computer science, design, education (elementary education, en¬ 
gineering education, math education, and physics education), engineering (aeronautical, cognitive, 
mechanical, and electrical), industrial and product design, informatics and computing, science and 
technology, and sociology. To further support a goal of enabling connections and conversations, 
seven design educators were invited from Centro Advanced Design Institute, CUNY, NT, MIT, Purdue 
University, and Syracuse University. These guests signed the Data Use Agreement, had full access 
to all materials, and also attended the Research-to-Practice workshop. 
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As demonstrated on the DTRS 10 website ( http://docs.lib.purdue.edu/dtrs/2014/ ), participants 
engaged with the data in a variety of ways (Adams and Siddiqui, 2016). Methodological approaches 
encompassed a full range of qualitative, quantitative, mixed methods, and critical theory approaches. 
Some approached the data top-down using apriori frameworks and others generated themes bottom- 
up. There were micro to macro level analyses as well as longitudinal studies and comparative studies 
across students and disciplines. Similarly, the unit of analysis ranged from students to coaches to 
artifacts (e.g., student work and course syllabi) and to interactions between students, coaches, and 
artifacts. Some researchers used a single data type (video, transcripts, syllabi, and student work), 
although most combined data types. Finally, most (61%) focused on a single disciplinary context 
while the remainder studied multiple disciplinary contexts. Of these, the most common comparisons 
included mechanical engineering, industrial design, and choreography. 

While there are many ways to organize symposium papers into themes, the session titles give an 
indication of research outcomes covering a landscape linking design thinking, cognitive and learning 
sciences, and communication and discourse: impact of coaching on student reasoning, knowledge 
handling and information sharing, modalities of discourse, multiple framings of design thinking, 
feedback and pedagogical content knowledge, identity and becoming, and comparing disciplinary 
cultures. Examples of design thinking frameworks used include: abductive reasoning, creativity, design 
cognition, design grammar, disciplinary knowledge, divergent and convergent reasoning, generative 
reasoning, human-centered and empathic design, informed designing, multiteam systems, norma- 
tivity and design values, and professional vision and design expertise. The book that resulted from 
the symposium also illustrates the ways DTRS 10 studies represent an interparadigmatic mix. As an 
example, a number of research teams were interested in discourse and interactions within design 
reviews, drawing from multiple (and sometimes conflicting) paradigms. Some focused on micro or 
frame-by-frame analyses (analyzing gestures, body orientations, and stances), some focused on ana¬ 
lyzing activity systems (interactions between humans and non-human agents such as artifacts), and 
some focused on analyzing discourse elements (noun and verb usage, design grammar) (Adams and 
Siddiqui, 2016). Finally, to continue the theme of integrative syntheses and connecting research-to- 
practice, authors were required to include connections to other papers (connections to motivations, 
frameworks, observations) and articulate implications for action (Adams, 2016a; Turns et al, 2014). 


IMPLEMENTING DATA SHARING 


The following sections describe key challenges and experiences associated with implementing 
the data sharing process including: (1) collecting the data, (2) the IRB backstory, and (3) managing 
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the dynamics of data access and emergent sharing. These were selected to highlight the unique 
aspects of this project such as the Data Use Agreement, the commitment to attend the symposium, 
and the sharing of qualitative and identifiable information. 

Collecting the data 

All data was collected from June 2013 to January 2014 in natural settings rather than controlled 
environments. In most cases, filming involved a single camera. Multiple cameras were available, but 
there was often insufficient warning to set multiple cameras in place before the start of a design 
review or the amount of physical space did not afford multiple camera angles. Only a subset of 
the data collected is included in the final shared dataset. Oversampling provided flexibility in mak¬ 
ing choices regarding video and audio quality, sufficient longitudinal data (having clean data over 
multiple reviews), sufficient number of cases to discern patterns and variations, equipment failures 
(video or audio), and removal of sensitive situations. It is important to note that two sensitive situ¬ 
ations did occur (a medical emergency and a situation in which a student not involved in the study 
was accidently discussed during a design review). These situations were deleted from the shared 
dataset. DTRS participants were provided with a technical report that provided details of the data 
(Adams and Siddiqui, 2014). In the technical report, study participants are identified by first names 
to assist DTRS participants in following conversations and internal references within the video data, 
as well as student level and discipline. Information such as last names, email addresses, and the in¬ 
stitutional context were removed or “blacked out” from all data (e.g., transcripts, work documents, 
syllabi, project briefs). After the symposium, DTRS participants were provided with a “de-identifying” 
guideline for removing identifiers for any publicly disseminated work. 

All data was collected following principles for research involving human subjects, and all indi¬ 
viduals visible and potentially identifiable in any data voluntarily provided their written consent 
to participate after receiving information (written and oral) about the details of the project. This 
occurred through a multi-stage process to ensure participants had many opportunities to ask 
questions, request to be removed, and make informed decisions regarding the risks and benefits 
of participation. The process began with contacting each instructor involved with a design review 
and explaining the study and consent process during a face-to-face meeting. All these instructors 
permitted a member of the DTRS 10 team to later visit their classrooms and present the study to 
their students, all of which were legal adults. Similar to the instructors, the first meeting with stu¬ 
dents involved explaining the study including: what data would be collected and how it would be 
shared, freedom to leave the project at any time, and a description of potential risks and benefits 
(e.g., the potential for private information and intellectual property (student design projects) to 
become public, systems for protecting confidentiality such as storing data in a secure password 
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protected system and requiring de-identifying techniques for any public dissemination, and not 
providing information on participation to instructors to minimize feelings of coercion related to final 
course grades). Students had opportunities to ask questions and indicate if they did not want to 
participate. There were very few questions and answers to questions were promptly offered. 

This was only the beginning of the consent process. A key part of the first presentation on 
the study was communicating that initial data collection would be based on initial consent and 
that the last day of data collection would involve a final consent process. On the final day of data 
collection, a member of the DTRS 10 team summarized the study emphasizing the data sharing 
aspects, potential risks and benefits, and processes for protecting privacy. There was time to read 
the consent form and ask questions. Participants were told that signing the form would indicate 
voluntary consent and leaving the form unsigned would indicate not consenting. In this way, all 
participants submitted a consent form (signed or not) and instructors would not be able to de¬ 
termine which students provided consent. All participants received a copy of a consent form that 
included information for contacting the team if they had additional questions or later changed 
their decision regarding participation. 

The rationale for this approach was to provide multiple opportunities over this longitudinal study 
for participants to choose to participate in or leave the study, but more importantly for participants 
to understand the nature of the data being collected to be able to assess the risks and benefits 
of their participation. For the team collecting the data, this can be risky since it involves investing 
in collecting data that must later be destroyed; however, given the goal of the shared data set it 
was critical to take precautions to ensure participation was based on informed decision making. 
All participants except one signed the consent form. Because that one person was part of a team, 
that team was also removed from the study. To date, there have been no additional requests to be 
removed from the study. This process was followed in all settings with the exception of the service¬ 
learning team. Because this data was collected on site, where there were legal minors, there were 
additional precautions to ensure that no minor was included in the data set. 

The IRB backstory 

As illustrated in the previous section, much of the Institutional Review Board (IRB) process 
for securing human subjects approval for this data sharing project followed standard procedures 
regarding people, processes, and protections. There were two significant divergences - sharing 
identifiable data and sharing data at this scale. Our IRB office had limited experience with sharing 
video recordings (which inherently involves identifiable data) at a global scale. This contributed to 
a highly iterative process fraught with misalignments. The first series of iterations involved requests 
to remove identifiers from the data. While blurring faces in digital videos was considered an option, 
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this was not done since it would conflict with the nature of some of the anticipated analyses (e.g., 
eye tracking, facial gestures, etc.), many of which are important research contributions in the field 
of design thinking. Rather, as described in the next section, the decision was to use a Data Use 
Agreement that required users of the dataset to follow specific procedures for protecting study 
participant confidentiality and privacy. As such, IRB approval included approval of sharing identifi¬ 
able data and the process for de-identifying data for public use. 

The second series of iterations, which also led to creating the Data Use Agreement, involved ef¬ 
forts to meet an IRB requirement of identifying all members of the research team on the participant 
consent form. In a traditional situation, identifying researchers who would have access to the data 
occurs as part of writing an IRB application as information communicated on the participant consent 
form. For DTRS 10, the timelines for submitting the IRB application and for committing to participate 
in the symposium could not be synergistic. Participating in the symposium involved submitting a 
letter of interest and then signing the Data Use Agreement, after which access to the dataset and 
the technical report was granted. In this way, DTRS participants had opportunities to evaluate the 
dataset for relevance and applicability to their individual expertise prior to submitting a formal letter 
of commitment. However, the IRB process required listing all possible researchers on the consent 
form as a condition for obtaining approval to collect data. Listing all potential researchers on the 
consent form would be on the order of 1000 words, listing about 240 people (e.g., 80 potential 
teams comprised of three people each) and their contact information, and would require constant 
IRB revision approvals as researchers joined or left the project. It would also likely be overwhelming 
to study participants. This further indicated the traditional IRB process would not be effective for 
this situation of a globally shared dataset. 

After three months of iterative revisions, the final solution was to create a Data Use Agreement 
process, which allowed IRB to approve the consent form with only the names of the researchers that 
collected the data. Our IRB office had little experience or advice to offer, particularly with agreements 
for sharing identifiable data such as video. In addition, at our campus the approval process for Data 
Use Agreements is under the purview of the Sponsored Program Office, which interacts with, but 
acts separate from, the IRB office. This is not a unique situation; a search on Data Use Agreements 
suggests that this may be common practice, and potentially a point of common concern where is¬ 
sues of monitoring researcher ethics and accountability could get lost within the space between 
these two administrative offices. Creating the DTRS 10 Data Use Agreement involved conducting 
a search for best practices and erring on the side of “highly conservative”. Example agreement 
forms targeted schools with medical programs (often the most stringent IRB guidelines) and best 
practices included (1) sharing of identifiable information, (2) evidence of following best practices 
for ethical human subjects research, (3) articulating limitations on data use as well as confidentiality 
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and obligations under breach of contract, and (4) language that would be legally binding. The draft 
Data Use Agreement was sent to our Sponsored Programs office where minor modifications were 
made and later approved. Once this was completed, the human subjects application was also ap¬ 
proved. As described earlier, the Data Use Agreement was an explicit part of the consent process 
communicated to all participants and that unethical behavior on the part of those who signed the 
agreement was a possible risk. 

Managing the dynamics of data access and emergent sharing 

The process of converting data into shareable formats and placing these in a secure server was 
relatively straightforward. This project uses existing platforms and data management systems. All 
the data reside on the secure password-protected Purdue University Research Repository (PURR). 
As shown in Table 1, data is stored using file extensions independent of application software, hard¬ 
ware, and operating systems (e.g., mp4, pdf). The most critical and time-consuming issue involved 
reducing digital videos in size (e.g., from 3GB to 300MB) for easy uploading and downloading. 

DTRS 10 participants who signed the Data Use Agreement were granted PURR access. This 
required users to agree to (1) use appropriate safeguards to prevent use or disclosure of data 
other than permitted by the agreement, (2) replace identifiers with pseudonyms and cite the 
description of the database for any presentations or publications, (3) send any product that in¬ 
cludes the use of the database (e.g., reports, dissertations, papers, presentations) to the owners 
of the database, and (4) cease use and destroy the data if the agreement is terminated. Similarly, 
it required users to agree NOT to (1) use the data to create any commercial product or generate 
profit, (2) distribute the data to third parties or websites, and (3) contact the individuals whose 
information is contained within the database. Noncompliance was articulated as a breach of 
contract that would result in termination of access. By signing the Data Use Agreement, DTRS 


Digital video Videotapes of design reviews, reduced to a resolution similar to YouTube for easy downloading from the 
recordings PURR system. 

File extension: .mp4 (digital multimedia). May be converted to other formats for viewing and editing. 

Transcripts Written as verbatim and time stamped at each minute. 

File extension: .doc (Microsoft Word). May be converted to other formats such as rich text format (.rtf) 
for importing into various software analysis programs. 

Artifacts Work reports, presentations, written feedback, and course / project syllabi. Videotapes of draft 

choreography performances are provided as works-in-progress referenced in the choreography reviews. 
Due to the scale of this study, sketches and prototypes used during review sessions are not included as 
separate documents; rather, videotapes were filmed to provide adequate detail. 

File extensions: .pdf (portable document format) and .mp4 (digital multimedia) 


Table 1. An overview of the DTRS JO data file types. 
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participants agreed to follow guidelines for protecting privacy and confidentiality in any products 
made public such as using pseudonyms for study participants and their context and blurring 
images. Participants received a guide for removing identifiable information including a list of 
approved pseudonyms. These agreements are being managed and monitored within the process 
of publishing symposium materials on the website, and creating the edited book and edited 
special issues for Design Studies and CoDesign. For example, all DTRS 10 related publications 
have been reviewed multiple times by an internal committee to confirm identifiable information 
was removed. Similarly, researchers who signed the Data Use Agreement are asked to provide a 
copy of any recently published work. Although we have not encountered breaches of conduct, 
the mechanism for holding individuals accountable for breaches of conduct involves reporting 
researchers to research integrity and ethics offices at home institutions as well as documenting 
the breach at the host institution. Monitoring this is an ongoing challenge that requires integrating 
IRB and Data Use Agreement systems as well as advancing institutional policies for monitoring 
and sanctioning research ethics. 

Another unique aspect of this data sharing project are the mechanisms for updating the data¬ 
set in response to the needs of DTRS 10 researchers. As researchers accessed the data, questions 
emerged such as availability of course syllabi or project briefs, sequencing of design reviews (such 
as which student or coach went first, second, third, etc.), and design competition outcomes. Discus¬ 
sion boards and messaging systems were used to log and respond to questions so that all users had 
access to new information, some of which involved updating the dataset. For example, questions 
about the availability of course syllabi involved asking study participants if they would be willing to 
share these and confirming that the sharing of syllabi was covered within the original consent form 
as “work documents”. With permission, all syllabi were uploaded to PURR with identifiers removed 
such as institutional context, course number, names, phone numbers, and email addresses. This 
was followed with a post to the discussion board and a pointer to the updated technical report. As 
another example, one researcher assumed that the industrial design program was associated with 
a college of engineering, and another researcher clarified this perception, noting that the industrial 
design program was associated with a college of liberal arts. 

The PURR cyber-infrastructure also became a place for sharing intermediate works-in-progress 
such as research proposals. This allowed participants opportunities to engage early and often in 
making connections across research studies and generating ideas for the Research-to-Practice work¬ 
shop. For example, the review process was handled through the Purdue e-publication system where 
participants engaged in two feedback cycles. They were asked to provide guidance on improving 
research quality, make connections to their own study, and generate ideas for research-to-practice 
connections. 
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FROM SHARING TO LEARNING PARTNERSHIPS 


For DTRS 10, the design of the shared dataset and the design of the symposium experience 
were intentionally synergistic. Here, the goal was to broaden the data set experience from sharing 
data to enabling learning partnerships (Baxter Magolda and King, 2004) focused on the nature and 
nurture of design thinking. This was an effort to move beyond information dissemination models to 
transformative learning approaches to educational transformation (Siddiqui and Adams, 2013). Ac¬ 
cording to transformative learning theory (Mezirow, 2000), through critically assessing the relevance 
of existing assumptions individuals can reframe their values and perspectives, and bring them into 
greater alignment. For this project, the goal was to support alignment between theory and practice 
in designing learning environments, across research findings towards an integrative synthesis, and 
within an individual as integrating multiple identities as researcher, teacher, and designer. 

Discourse among members of a community plays an important role in transformative learning and 
learning partnerships. To catalyze conversations during the symposium we used artistic inquiry and 
performance art to support critical reflection and empathic awareness. To catalyze conversations 
after research presentations, the final day of the symposium was a Research-to-Practice workshop 
with a goal of empowering DTRS participants to develop an enduring perspective for integrating 
research of design thinking and teaching practice along interpersonal, intrapersonal, and epistemo¬ 
logical dimensions. Key features of the artistic inquiry and workshop activities are provided below. 

Artistic inquiry as a tool for epistemic re-structuring 

Artistic inquiry emphasizes the role of the imaginative intellect in creating, criticizing, and con¬ 
structing knowledge that is not only new but also has the capacity to transform human understand¬ 
ing (Sullivan, 2010). Dance and movement arts are also useful in developing critical reflection and 
somatic awareness (i.e., sensory, proprioceptive, and kinesthetic knowledge) (Brightman and Jaycox, 
2011; Jaycox et al, 2014; Jaycox and Brightman, 2012). For DTRS 10, artistic inquiry was translated 
from a theatre venue to a conference setting as a tool for encouraging epistemic re-structuring and 
disrupting the cognitive habitus of conferences (McMullen, Brightman and Jaycox, 2016). The goal 
was to activate perspective shifts among symposium participants and to invite novel insights about 
the symposium theme by creating moments of unfamiliarity, surprise, and unexpected ambiguity. 
This involved a four-part multicomponent progression that began with movement exploration intro¬ 
duced at a pre-symposium Embodied Feedback workshop to provide some common experience of 
and language for body movement. The presence of performance was increased the following day 
through short, silent vignettes during breaks. Performers portrayed gestures, which mirrored those 
observed in the shared dataset, as interactive social artifacts and metaphors to help participants 
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step out of their current mindset and step into an empathic awareness about the ways feedback is 
embodied and experienced in design review conversations. 

The main performance event, Gestural frequencies , was presented in the afternoon. A post¬ 
performance dialogue occurred the following day to allow the extended period of ambiguity sur¬ 
rounding the performance art approach to begin to resolve and be assimilated into wider views 
on analyzing design review conversations. A series of prompting questions were introduced (e.g., 
experiences and engagement with the performances, aspects of the performances that were 
challenging) and the audience was encouraged to speak to each other about their experiences. 
This dialogue was audiotaped and later analyzed. Overall, the summative feedback revealed that 
disruption was experienced along multiple dimensions - from the emotional and visceral (e.g., 
feelings of disturbance, frustration, and delight) to the reflexive (e.g., shifting perceptions over 
the cumulative experiences and metacognitive moments of “made me think” or “personal grow¬ 
ing moment”). Details of these activities and their outcomes are provided elsewhere (McMullen, 
Brightman and Jaycox, 2016). 

Collaborative inquiry within a Research-to-Practice workshop 

A Research-to-Practice workshop on the final day provided opportunities for DTRS participants 
to build on conversations from the research presentations to collaboratively develop teaching re¬ 
sources that could help students develop as design thinkers (Siddiqui, Adams and Fosmire, 2016). 
Seven design educators were specifically invited to attend as research-to-practice “facilitators”; 
most if not all of the DTRS 10 participants are also design educators. The workshop goals were to 
empower participants to: identify practical problems of design learning and teaching from personal 
experiences, use research to collaboratively conceptualize these problems, use a curricular model to 
develop actionable plans to address a problem, and co-construct new perspectives about integrating 
research and teaching. A collaborative inquiry framework (Kasl and Yorks, 2002) was used to design 
workshop activities that could support inquiry into bridging research and practice and align with 
a goal of transformative learning that is the essence of learning partnerships. Cycles of reflection 
and action focused on multiple ways of knowing: personal (drawing on experience to individually 
and collectively identify design learning problems), propositional (conceptualizing problems using 
research supported concepts), and practical (developing context-specific actionable plans). Par¬ 
ticipants were encouraged to use a generalized curricular model (NRC, 2001; Streveler, Smith and 
Pilotte, 2012) to develop these plans. This model has two central principles: (1) instructional design¬ 
ers needs to be explicit about the ideas that shape the learning environments they create, and (2) 
effective learning environments are holistically integrated or coherent - with alignment between 
learning goals, learning experiences, and evidence of learning. 
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The evaluation of the workshop included analyzing summative feedback and artifacts produced 
during the workshop such as posters synthesizing design teaching and learning challenges or 
goals, and written actionable research-to-practice teaching plans for a specific teaching context. 
Discussions throughout the workshop revealed how participants experienced a continual research- 
to-practice cycle that included identifying new research questions to explore in the dataset, and 
appreciated the explicit activities focused on finding coherence among goals-experiences-evidence 
of learning. Many spoke to the value of having extensive opportunities to reflect (individually and 
collectively), integrate, and deepen discussions over the course of the symposium as a whole. 
Details of these activities and their outcomes are provided elsewhere (Siddiqui, Adams and 
Fosmire, 2016). 

Broadening the DTRS 10 data sharing experience 

New knowledge about design thinking and how it can effectively be developed in students 
has emerged from this data sharing event. Information on the DTRS 10 design and outcomes is 
available through electronic proceedings ( http://docs.lib.purdue.edu/dtrs/ ), conference papers 
(Adams et al, 2014), an edited book (Adams and Siddiqui, 2016), and two special journal issues 
(forthcoming in 2016). The theme of the Design Studies special issue is on connecting design in¬ 
quiry, design being, and design coaching (Adams, Cardella, and Purzer; 2016). The theme of the 
CoDesign issue is on visualizing design interactions emphasizing the participatory and collabora¬ 
tive exchanges that underpin design reviews as sites of contingency, negotiation, mediation, and 
multidirectionality (Adams, McMullen, and Fosmire, 2016). As a collection, the outcomes of DTRS 
10 may be characterized as challenging existing paradigms, opening up new lines of research ques¬ 
tions (such as questions about designerly ways of being and embodied designing), integrating 
frameworks across disciplines, and making comparisons across disciplines that reveal how much 
different disciplines can learn from each other. Given the educational transformation agenda, 
the book provides summary tables of outcomes in terms of theoretical and research-to-practice 
connections and contributions (Adams, 2016a). 

An emerging story is how this project is catalyzing next generation data sharing activities as well 
as contributing to new global data sharing projects. Next generation projects are those in which 
new users are granted access to the current dataset or existing users conduct new or follow-on 
studies. Next generation projects that broaden access to the dataset are being managed in similar 
ways to the DTRS 10 event. First, a researcher needs to have knowledge of the dataset, which seems 
to be occurring through social networks among DTRS 10 participants or through the original call 
for participation. The next step is to contact the first author of this paper and submit a statement 
of intent. If this statement aligns with the goals of DTRS 10 and is within the boundaries of the 
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IRB and Data Use Agreement regarding the ethical treatment of human subjects, then the new 
users follow the same process as the DTRS 10 participants (e.g., sign the Data Use Agreement and 
all that it represents including updates on data use and publications or presentations). Examples 
of next generation projects include providing access to graduate students in a design thinking 
course as a shared research experience and to individuals who submitted a DTRS proposal but 
were unable to attend. Some of these projects have already been published. A long-term goal is 
to use the public DTRS 10 website as a central home for archiving data sharing products from the 
DTRS 10 dataset. 

The DTRS 10 model has also catalyzed new data sharing projects. This provides opportunities 
for understanding how this experience may translate to new contexts. For example, DTRS 11 will 
continue a shared dataset history and build on the design principles and lessons learned from 
DTRS 10. The authors are also exploring ways to repeat DTRS 10, working within annual conference 
structures within the engineering education community. This experience is also helping shape new 
ventures such as a global collaboration around a real-time “wide field ethnography” data sharing 
project in the context of a software design firm (Socha, Adams, Franznick, Roth, Sullivan, Tenenberg 
and Walter, 2016). All of these are providing opportunities to learn from and attend to some of the 
challenges of DTRS 10, explore new affordances of technology (capturing, storing, sharing, and 
collaborating), deepen a collective understanding of design principles for global data sharing for 
educational transformation, and new research into collaborative inquiry around shared datasets 
(Socha, Jornet, and Adams, 2016). 


SUMMARY 


The previous sections describe experiences with designing and using the dataset, obtaining IRB 
approval, managing access to the data, enabling dynamic updates to the dataset, and moving from 
shared data as an individual activity to a learning partnership experience. In this section, we draw 
on these experiences to identify recommendations regarding large scale data sharing and principles 
for broadening data sharing to include an educational transformation agenda. 

This DTRS 10 experience of designing a shared dataset to catalyze connections across perspectives 
on the nature and nurture of design thinking has made visible a variety of challenges with sharing data 
globally to advance educational transformation. The following is a summary of recommendations for 
implementing future large scale data sharing projects. 

• Adopt a user-centered perspective. While many of the aspects of data sharing fall within 
current practices, many do not. It is important to engage various stakeholders (study 
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participants, data users, data management designers, human subjects policy makers and 
managers, etc.) early and often, adopting an iterative human-centered approach to design¬ 
ing data sharing systems. 

• Build on prior art. It is important to remember that many features of the DTRS 10 design were 
based on lessons learned from prior work, broadly conceived. This was both generative and 
freed up cognitive and collaborative resources to design for intention rather than expend 
limited resources attending to the considerable administrative load associated with managing 
the event. 

• Look for and attend to misalignments across the data sharing system. While most 
misalignments highlight critical revisions, they also highlight opportunities for making system 
improvements that enable accountability between stakeholders that support sustainable 
ethical practice. 

• integrate Data Use Agreements into the human subjects review process. Having these agree¬ 
ments approved outside of a human subjects approval process could make invisible mechanisms 
for holding individuals accountable to breaches of conduct. Much could be learned from a 
variety of professions (e.g., health science, medicine, anthropology, qualitative researchers) 
regarding large scale data sharing of sensitive and identifiable data. An integrated plan needs 
to offer clear mechanisms for monitoring Data Use Agreements as well as mechanisms and 
sanctions for resolving violations. 

• Enable wide field science. Advances in technology are increasing opportunities for wide field 
science (Socha et al, 2016a; 2016b), however these are outpacing advancements in manag¬ 
ing, indexing, navigating, and making large data sets accessible and available for action. As 
a small example, data files for DTRS 10 had to be significantly compressed (from 3-5 GB to 
300-400 MB) to allow ease of access. Doing this for a few files is not an issue; doing this for 
86 files is non-trivial. Similarly, uploading and downloading 300 individual data files (video 
recordings, transcripts, work products) is non-trivial, potentially reducing participation and 
use. Future software and platforms for data sharing need to address issues from the simple 
(data compression and efficiency in downloading and uploading) to the complex (tools for 
indexing, navigating and annotating data from a collaborative inquiry perspective). 

• Be open to the ways data sharing can create learning partnerships. A shared dataset is a 
dynamic, shared, cognitive object. It is not just about access; it is also about interaction, 
dialogue, and knowledge creation. Data sharing platforms that provide mechanisms for 
interaction and emergence can have far reaching potential. 

Within an educational transformation agenda, this DTRS 10 experience also offers principles 
for intentionally designing for data sharing and moving from data sharing to enabling learning 


18 


SPRING 2016 





ADVANCES IN ENGINEERING EDUCATION 

Designing for Global Data Sharing, Designing for Educational Transformation 



partnerships. The following offers a language for contributing to a broader conversation on design¬ 
ing for global data sharing: 

• Take a complex systems approach. The intentional design of the shared dataset was part of 
a system of ideas. We drew on relevant theory as first principles that guided the shape and 
direction of the design, and iterated across elements (the dataset, artistic inquiry activities, 
Research-to-Practice workshop, cyber-infrastructure, and even the social events) towards a 
holistic integration. Connecting the various elements was a goal of enabling opportunities for 
people to engage in critical reflection, discourse, and dialogue. 

• Design for emergence. This means being inclusive, providing multiple entry points, and provid¬ 
ing a learning environment that allows different perspectives or paradigms to thrive, collide, 
synergize, and self-organize. The call for participation was inclusive (oldtimers and newcom¬ 
ers with diverse backgrounds and disciplinary homes), and decisions about the contents of 
the dataset were shaped by a multiple perspective methodology (Linstone et al, 1981) and 
complexity theory (Davis and Sumatra, 2006). 

• Select a phenomenon for study that lives in a middle space between research and practice. A 
phenomenon that is familiar and relevant (what Lysaker and Furuness (2011) describe as “real life 
work”) to practitioners as well as significant and theoretically rich to researchers creates potential 
for triggering a reciprocal flow of dialogue along the research-to-practice cycle. This has potential 
to advance knowledge in both domains (see ASEE, 2012) - general knowledge grounded in research 
and practical knowledge grounded in particular cases of problem solving. It also sets up a learning 
cycle where general knowledge is used to interpret a particular case, the particular case yields new 
information that may be used to modify the general knowledge, and this in turn leads to different 
ideas about how to respond to a particular case, and so forth. This affords a continual reinvestment 
in learning that enables generating new knowledge as well as applying old knowledge in new ways. 
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